Desktop sharing has become an important feature in current collaboration software. It allows virtual meeting attendees to be viewing the same material or content (video, documents, etc.) during a discussion. To make desktop sharing possible, the screen content that is being shared by the sending computing device during a collaboration session must be continuously captured, encoded, transmitted, and finally rendered at receiving computing devices for display.
A hybrid video encoder is the most popular and the state of the art video encoder in current video coding. It consists of three major parts, predictive coding, transform coding and entropy coding. In predictive coding, a coding tool called inter-frame prediction (or motion-compensated prediction) is found to be efficient in compression due to the high temporal correlation in video signals. An inter-frame is a frame in a video compression stream that is expressed in terms of one or more previously decoded frames. Inter-frame prediction takes advantage of temporal redundancy between previously decoded frames to achieve higher compression efficiency in comparison to using neighboring pixels in the same frame. A search module is used to ensure the efficiency of inter-frame prediction, where a search can be conducted to find a decent predictor block of pixels that is useful for coding the current block of pixels.
Choosing an appropriate search module for a hybrid video encoder is important since the search result will substantially affect the compression efficiency of the encoder, and further since the cost of the search scheme itself can be computationally expensive. Two types of search schemes that are typically utilized are a full search and a fast search.
A full search is a method that examines all the candidate positions in a search window (i.e., a portion of a reference frame) to find the best predictor for inter-frame prediction (where the best predictor is the predictor that can produce the least residual after inter-frame prediction). The full search is a complicated and exhaustive search that may provide the best inter-frame prediction but also requires the most computations and is too slow for many real-time applications.
A fast search is a method of checking selective candidate positions in the search window to find the best predictor. Some examples of known fast searches include a diamond search, a hexagonal search, etc. The fast search can provide a fast computation. However, the performance of the fast search is typically not as reliable as the full search in that it presents the potential risk of missing the best predictor.
The search module algorithm is significant to encoding efficiency when a hybrid video encoder is applied to encode screen content. For example, when a hybrid video encoder is taking screen content as input, inter-frame prediction can be exploited to reduce inter-frame redundancy in displaced content. For newly appearing content, the search can also be used to find similar content as the predictor. One feature of screen content video that has been determined by observation is that screen content video typically contains similar patterns which can be used in prediction. For newly appearing content, when the predictor can lead to efficient inter-frame prediction, bit-consumption can be greatly decreased in comparison to intra-frame prediction (which can be inefficient for some screen content).
The overall screen content resolutions can be significantly large, such that a full search calculation would be very complex and time consuming. Thus, fast searching is desirable over full searching for encoding screen content. However, conventional fast search methods are designed based upon camera video features rather than screen content features. Utilizing such conventional fast search methods for screen content can yield undesirable results.
A desirable search scheme for encoding screen content during desktop sharing should be able to find a reasonably good predictor for inter-frame prediction for a current block of content while not requiring a significant number of computations.